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Evaluation of UlI-sponsored Training 


Norman Park, Craig Riddell and Robert Power 


Purpose 


The purpose of this brief is to present a summary of 
a major evaluation of Ul-sponsored training in 
Canada between 1988 and 1991. While previous 
studies have focused on the effects of particular 
training programs on participants, this evaluation is 
unique in presenting a comparative analysis of the 
training options available to UI recipients and their 
various impacts on labour market outcomes. 


Introduction and context 


Heading into the mid 1990s, Canada faces 
numerous well-publicized problems of economic 
adjustment, ranging from industrial restructuring in 
response to competitive pressures to high 
unemployment rates. The number of jobs requiring 
a high level of skills is increasing, and many jobs 
that traditionally required lower levels of skill are 
disappearing. A mismatch between available and 
required skills means that some jobs go unfilled 
despite persistently high unemployment rates. For 
these reasons, government policies that promote 
job-related training are receiving more attention in 
Canada. 


Job training may be particularly beneficial for 
people on UI. Indeed, a significant number of the 
unemployed may not be able to find work without 
further training. If true, this calls into question the 
traditional view that the function of UI is to provide 
temporary income replacement until the claimant 
can find work. More active steps such as enrolling 
in training programs may be necessary for a 
claimant to become re-employed. 


These considerations prompted the federal 
government to announce the Labour Force 
Development Strategy in 1989. In this policy, the 
government stated that it planned to shift the 
emphasis of the Unemployment Insurance program 
toward more active measures such as training and 
away from passive income support. Subsequently, 
with the passage of Bill C-21 in November 1990, a 
greater percentage of the Unemployment Insurance 
fund could be earmarked for Developmental Uses 
such as training, in support of its objective “to 
provide an earlier return to employment by 
improving the employability of UI claimants 
through training.” 


This evaluation initially focused on a particular 
training option for UI claimants, the Feepayer 
training option. Under this option, claimants may 
receive UI and are exempted from having to search 
for a job while taking training, although they or a 
third party must pay the cost of the training. It soon 
became evident, however, that a broader and 
comprehensive evaluation of UI-supported training 
could be carried out at the same time. This involved 
the assessment of the Feepayer training program as 
well as four other training programs taken by UI 
claimants. This study examined the following 
training programs: Feepayer, DIR! and three 
programs under the Canadian Jobs Strategy (CJS) 
— Job Development, Job Entry, and Skill 
Shortages. In addition, this study included a group 
of regular UI claimants for comparative purposes. 


Each CJS training program has been evaluated in 
the recent past; but none of the evaluations has 
analysed systematically key findings by receipt or 


1 DIR clients (section 14) are defined as UI claimants who take part-time training without EIC sanction. DIR clients 
may be disentitled for not being available for work, or may be permitted to remain eligible for UI if the training 
(which usually occurs in the evening) does not interfere with the job search. 
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non-receipt of UI. Thus, this evaluation 
specifically addresses whether government training 
programs enhanced the re-employment prospects of 
UI claimants and whether the programs differed in 
their effectiveness. 


The evaluation of UI training directly compares the 
relative effectiveness of the Feepayer, DIR, and 
CJS approaches in terms of five dimensions of 
employability: re-employment success, length of 
time to get a job, stability of employment obtained, 
receipt of social assistance, and earnings. It 
focuses on regular UI recipients, and compares 
outcomes of each training program to outcomes of 
a comparison group of non-trainees (i.e., regular UI 
claimants who did not take training while on UI). 


Evaluation methodology and data 


This evaluation uses a non-experimental design to 
estimate program effectiveness. This approach is 
required because applicants are selected into 
training programs and not randomly assigned as 
would be the case in a true experiment. A variety 
of econometric and statistical techniques are used 
to assess whether the five training programs have 
an incremental impact on employability as 
compared to a group of regular UI beneficiaries not 
receiving training. 


The study is based upon the experiences of UI 
trainees and non-trainees whose UI receipt 
commenced in one of four years — 1988, 1989, 
1990, or 1991. Inclusion of these different cohorts 
permits us to assess the effects of the business cycle 
on the impact of the training program. The first 
two cohorts represent UI clients who participated in 
training programs prior to the onset of the 
recession; the last two cohorts were in training and 
sought work during a recession. 


For many of the outcome measures of interest, we 
obtained longitudinal data, beginning two or more 
years prior to the start date of each cohort. This 
information is important because it permits us to 
determine the incremental impact of training 
programs by comparing the labour market 
outcomes before and after UI receipt, with or 
without training. 


To assess the impacts of the training programs on 
the employability of clients and to gain accurate 
knowledge of the selection process into training, we 
conducted: 


* areview of regional guidelines and operational 
procedures; 


* personal interviews with EIC regional officials 
in all 10 provinces; 


mail surveys of 2450 UI trainees and 927 non- 
trainees (random samples of those on UI from 
January 1988 to June 1991); 


¢ atelephone survey of 30 Canada Employment 
Centre managers; 


* amail survey of Canada Employment Centre 
counsellors across Canada; and 


* an intensive analysis of EIC administrative data 
on the trainees and non-trainees. 


Main findings 


To assess the relative effectiveness of training 
programs in improving employability of UI 
claimants, we evaluated success on five 
dimensions: the probability of re-employment, the 
length of time needed to get a job, the proportion of 
time spent employed, the receipt of social 
assistance, and earnings. 


Probability of re-employment 


The incidence of re-employment differed 
substantially across training programs and cohort 
(year on UI). Controlling for demographics, work 
history and environmental characteristics, 
participants in the Feepayer and Skill Shortages 
programs were no more likely to become re- 
employed than the UI only group, while DIR, Job 
Development, and Job Entry trainees were 
generally significantly less likely than UI only 
claimants to obtain a job after Ul/training. 


Various traits and circumstances of the clients had a 
significant impact on the chances of re- 
employment, but the effect of these traits varied by 
cohort. Thus older workers had a lower chance of 
obtaining a job after UI. Region was generally 
insignificant, although those in B.C. and Alberta 
were significantly more likely to locate 
employment than Ontario clients. Gender had no 
effect in any year, and education had a positive but 
generally insignificant effect on the likelihood of 
finding a job. Having children under six 
significantly reduced the incidence of re- 
employment, as did being an immigrant, an 
aboriginal, or a visible minority. Re-employment 
prospects were higher for those who lost their job 
because they moved, returned to school, quit, or 
had seasonal work; not surprisingly, those who 
were on temporary lay-off were much more likely 
to become re-employed; job tenure, part-time work, 
and union membership had no effect. 


It is important to emphasise that this cross-sectional 
analysis cannot determine the extent to which these 
differences are a consequence of the training 
intervention and to what extent they reflect 
selection into the programs (i.e. unobserved factors 
which make participants in the DIR, Job . 
Development and Job Entry less likely to become 
re-employed than their UI only counterparts, even 
in the absence of training). 


Duration of search to first job 


Trainees required substantially less time than non- 
trainees to find a job following training/UI. For 
those who became re-employed, we found large 
differences between trainees and non-trainees in the 
length of time required to obtain a job following 
training/UI, with trainees requiring 11-17 weeks 
less (after completion of training) to become re- 
employed than non-trainees. More education 
reduced the amount of time it took to find work 
(about three weeks for every extra year completed). 
Provincial differences were small. Those who 
expected to be recalled to work within six months 
were re-employed six to seven weeks earlier than 
those who did not expect to be recalled. No other 
explanatory variable was significant. 


Whether these differences in the length of time to 
find a job are a consequence of the training 
intervention or a reflection of selecting into training 
individuals with unobserved characteristics that 
make them more likely to become re-employed 
quickly cannot be determined from our cross- 
sectional analysis of the post-training period. 


Proportion of time spent employed 


Data on the proportion of time spent employed 
were available for both the pre- and post-training/ 
UI periods. Thus it is possible to take account of 
the finding that training programs attract clients 
with very different average levels of time spent 
employed in the absence of training. In contrast to 
our findings relating to the probability of re- 
employment and the duration of non-employment, 
our estimates of the impact of training on the 
proportion of time spent employed take account of 
selection into the programs, and can thus be 
attributed to the training intervention. Concerning 
proportion of time spent employed, the 1988 cohort 
experienced the largest gains (post- versus pre-UI) 
followed by the 1989 cohort: the recession may 
well have lowered the proportion for more recent 
cohorts. clients in B.C., Saskatchewan, and Nova 
Scotia experienced significantly larger increases in 
time employed than those in Ontario. Of the other 


control variables tested, only tenure in the previous 
job had a significant effect: those with less tenure 
experienced greater gains. Controlling for client 
characteristics, we found that only the Job Entry 
program had a significantly positive impact on 
proportion of time employed. The Feepayer, Skill 
Shortages and DIR programs also generally had 
positive, but insignificant, impacts. 


Reliance on social assistance 


Training may be expected to reduce reliance on 
social assistance relative to the period before 
training. In comparing pre- and post-training 
periods, relative to non-trainees a modest but 
significant rise in the incidence of welfare receipt 
was observed for Job Entry and Job Development 
clients. The increased need to rely on social 
assistance was primarily restricted to Ontario; this 
is consistent with the view that the rise in social 
assistance during our sample period is associated 
with the 1990-92 recession, which hit Ontario 
particularly hard. Men, unmarried persons, and 
those with longer job tenure prior to UI experienced 
significantly larger rises in reliance on welfare. 


Earnings 


Examination of descriptive statistics revealed that 
different training programs attracted individuals 
who had very different average earnings prior to 
training. Relative to the UI only group, Feepayer 
and DIR participants had similar average annual 
earnings (prior to training), Skill Shortages 
participants had higher earnings, and Job Entry and 
Job Development clients earned less. Reinforcing 
the time series evidence on earnings differences, 
evidence from our regional interviews and 
counsellor survey showed that selection into the 
five training programs was non-random; rather 
selection depended on client traits, regional and 
local policy, cohort, and other factors. These 
observations suggest that it is important to account 
for factors that may cause earnings to differ, 
especially if those factors also influence selection 
into training programs. We did this by using the 
longitudinal nature of the earnings data, which 
allows one to control for unobserved factors that 
cause the earnings of some individuals or groups to 
differ from those of others. Sensitivity analysis 
showed the results were robust. 


The findings vary by cohort and the years used to 
measure earnings changes. For the 1988 cohort, 
the Skill Shortages, Feepayer, and Job Entry 
programs had significantly positive impacts on 
earnings; impacts for DIR and Job Development 


were positive but not significant. For the 1989 
cohort, only the Job Entry program resulted in 
significant impacts on earnings. Specifically, the 
estimated real earnings gains between two years 
before training and 1991 were: 


Estimated Estimated 
Real Earnings | Real Earnings 
Gain 1988 Gain 1989 

Cohort Cohort 
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Skill Shortages $6188 


Results may have been different for the two cohorts 
because the 1989 cohort entered the job market 
during a recession. 


$4054 


We next included the basic demographic variables 
to account for differences in the composition of 
groups participating in the various programs. The 
overall pattern of results held, although the 
estimated impact of training was somewhat lower, 
as some of the impact was attributable to client 
traits. These estimates implied that training 
impacts were larger (though not always 
significantly so) for those who were unmarried, had 
more education, and lived in B.C. Younger 
participants also benefited more than older ones. 


Rate of return 


To analyse the payback to the social investment in 
UI training, average real earnings change before 
and after training are estimated for the 1988 cohorts 
by using the UI only group as the control, and 
compared to the average real program costs, 
including expenditures on UI benefits and training 
course and allowance costs. 


When program costs are considered for the 1988 
cohort against their earning changes in 1991, the 
private rate of return in earnings per dollar of the 
original social investment was greatest for Feepayer 
Trainees (0.46), followed by Skill Shortages (0.06) 
and Job Entry (-0.29). This represents the return 
for a single year of earnings following training. 

The payback to trainees may be larger if cumulative 
earnings are considered against the training 
investment. 


Conclusions 


Were UI recipients better off as a result of having 
taken training? The answer depends on the 
outcome at issue and, especially, on the type of 
training. Judging by earnings — the most widely 
used, and arguably most important, index of 
training program impact — UI training programs 
effects ranged from zero (DIR and Job 
Development) to impressively large (Job Entry, 
Feepayer, and Skill Shortages), especially for the 
1988 cohort. Further, the most effective training 
appears to be training for a specific job, regardless 
of program label. 
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